Apparatus and method for selectively outputing image frames

ABSTRACT

An apparatus and method for selectively outputting image frames are provided. The apparatus includes a main topic frame extractor which extracts a main topic frame, which expresses a main topic selected by a user, from continuous image frames; a composite image frame creator which extracts a key object to realize the main topic from the main topic frames, and estimates a background object with respect to the key object to create a composite image frame using the key object and the background object; a wide-angle image frame creator which creates a wide-angle image frame, which includes the composite image frame and an image frame having common information to the composite image frame, among the continuous image frames; an attractive region setting unit which sets an attractive region with respect to the wide-angle image frame; and an output unit which outputs the attractive region.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority from Korean Patent Application No.10-2006-0022875 filed on Mar. 10, 2006 in the Korean IntellectualProperty Office, the disclosure of which is incorporated herein byreference in its entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

An apparatus and method consistent with the present invention relate toselectively outputting image frames, and more particularly, an apparatusand method for selectively outputting image frames, which can create awide-angle image frame emphasizing a main topic which a user selectsfrom continuous image frames, and select and output regions, which auser is interested in, from the wide-angle image frame.

2. Description of the Prior Art

Generally, in order to accurately reproduce the color of an imagecreated by a source device in an output device, color enhancementtechnology, which enhances brightness, saturation, and hue of the image,is used in an output apparatus. Such color enhancement technology can beapplied to an output consisting of moving pictures. Recently, theinterest has increased in the output of moving pictures and thecorresponding technologies for outputting high-definition movingpictures.

In the conventional art, U.S. Pat. No. 6,298,145 discloses technologyfor automatically extracting an image frame adjustably printed by usinga face detector, a blurring detector, or a motion analyzer. U.S. Pat.No. 6,393,163 discloses technology for correcting and displaying mosaicimages so as to be adjusted for a display system after the mosaic imagesare created using common displaying information. U.S. Pat. No. 6,724,937discloses technology for determining image frames, which calculates thevalue of a single scalar gradient showing characteristic of each framefrom continuous image frames, and seeks a gradient peak from smoothingdata. U.S. Pat. No. 6,940,526 discloses technology for selecting andcomposing two images after deciding a boundary by composing a backgroundimage and a main image.

The conventional arts as described above merely refer to technologiesfor extracting a key frame in order to output moving pictures, composedifferent image frames, or create mosaic images.

Therefore, there is a problem in that the conventional arts do notreflect a user's demands for the output of images.

Specifically, the conventional arts do not contain technology forselectively outputting a topic that interests a user.

SUMMARY OF THE INVENTION

Exemplary embodiments of the present invention overcome the abovedisadvantages and other disadvantages not described above. Also, thepresent invention is not required to overcome the disadvantagesdescribed above, and an exemplary embodiment of the present inventionmay not overcome any of the problems described above. Accordingly, anaspect of the present invention provides an apparatus and method forselectively outputting image frames, which allows a user to select amain topic he/she is interested in from continuous frames, and creatinga wide-angle image frame comprised of image frames to which the maintopic is reflected, thereby processing and outputting only the image inthe region selected by the user.

According to another aspect of the present invention, there is providedan apparatus which selectively outputs image frames, the apparatusincluding a main topic frame extractor which extracts a main topicframe, which expresses a main topic selected by a user, from continuousimage frames; a composite image frame creator which extracts a keyobject to realize the main topic from the main topic frames, andestimates a background object with respect to the key object by usingperipheral frame sequentially adjacent to the main topic frame, so as tocreate a composite image frame using the key object and the backgroundobject; a wide-angle image frame creator which creates a wide-angleimage frame, which includes the composite image frame and an image framehaving common information to the composite image frame, among thecontinuous image frames; an attractive region setting unit which sets anattractive region with respect to the wide-angle image frame; and anoutput unit which outputs the attractive region.

According to another aspect of the present invention, there is provideda method of selectively outputting image frames, the method includingextracting a main topic frame, which expresses a main topic selected bya user, from continuous image frames; extracting a key object, whichrealizes the main topic, from the main topic frames, and estimating abackground object with respect to the key object by using peripheralframes sequentially adjacent to the main topic frame, so as to create acomposite image frame by using the key object and the background object;creating a wide-angle image frame including the composite image frameand an image frame having common information to the composite imageframe, among the continuous image frames; setting an attractive regionwith respect to the wide-angle image; and outputting the attractiveregion.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects of the present invention will be moreapparent from the following detailed description taken in conjunctionwith the accompanying drawings, in which:

FIG. 1 is a block diagram illustrating the entire configuration of anapparatus for selectively outputting image frames according to anexemplary embodiment of the present invention;

FIG. 2 is a view illustrating processes for creating a composite imageframe according to the exemplary embodiment of the present invention;

FIG. 3 is a view illustrating processes for creating a wide-angle imageframe according to the exemplary embodiment of the present invention;and

FIG. 4 is a flowchart illustrating processes for selectively outputtingimage frames according to the exemplary embodiment of the presentinvention.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS

Details of additional embodiments are included in the detaileddescription and drawings.

Various aspects of the present invention will now be described morefully with reference to the accompanying drawings, in which exemplaryembodiments of the invention are shown. However, the scope of thepresent invention is not limited to such exemplary embodiments and thepresent invention may be realized in various forms. The exemplaryembodiments described below are provided to properly disclose thepresent invention and assist those skilled in the art to understand thepresent invention. The present invention is defined only by the scope ofthe appended claims. Also, the same reference numerals are used todesignate the same elements throughout the specification.

Hereinafter, an apparatus and a method for selectively outputting imageframes according to the exemplary embodiments will be described withreference to block diagrams and flowcharts in the accompanying drawings.

FIG. 1 is a block diagram illustrating the entire configuration of theapparatus for selectively outputting image frames according to anexemplary embodiment. Referring to FIG. 1, the apparatus 100 forselectively outputting the image frames includes a main topic frameextractor 110, a composite image frame creator 120, a wide-angle imageframe creator 130, an attractive region setting unit 140 for setting anattractive region in the wide-angle image frame, and an image outputunit 150 provided with an image processor 151 and a resolutioncontroller 152.

First, when a user selects a main topic, which is to be preponderantlyoutput, from continuous image frames, the main topic frame extractor 110extracts the main topic frame which has the selected main topicreflected in a ratio exceeding a reference ratio, from the continuousimage frame.

Here, the main topic may be selected from the entire content expressedin the continuous image frames, and may particularly include a portraittopic expressing a human, a landscape topic expressing a scene ofnature, etc. a structure topic expressing buildings or houses in a civiccenter, and an animal topic showing animals. For example, when theportrait topic has been selected by a user as the main topic, it ispossible to determine a reflection ratio of the portrait topic to themain topic, based on a ratio of an area occupied by a skin color of thehuman in the image frame in which the object expressing the portraittopic is displayed.

A frame in which the reflection ratio of the portrait topic is greaterthan the reference ratio can be extracted as a main topic frame. Thereference ratio may be a threshold of the area ratio that a key object(for example, portrait object exemplary expressed above) having ahighest reflection ratio of the main topic among objects expressed inthe main topic frame occupies, or a threshold of the color ratio thatthe skin color of the portrait object occupies.

For example, if the user selects the portrait topic as the main topicfrom the sequence of the continuous image frames in which landscapes andportraits are mixed, a face color and a face area of the selected humanare calculated so that the image frames, which have a greater face areathan the threshold value or a larger face color than the thresholdvalue, are extracted among the continuous image frames as the main topicframe. To the contrary, if a frame relating to the landscape is selectedas the main topic, the image frame in which the area or the color of theface region has the smallest proportion is extracted from the continuousimage frames as the main topic frame.

In order to extract the main topic frame, the threshold value is set byusing a cost function set in a unit block having a set size, and thenthe area or the color of the object expressed in the image frame can beestimated while increasing the size of a window by using an imagemasking method. In view of the area or the color of the estimatedobject, the main topic frame need not be a single frame. If thereflection ratio of the main topic is greater than the threshold value,plural frames can be set as the main topic frame.

After the main topic frame is extracted, a composite image frame creator120 extracts the key object from the main topic frame and then estimatesa background object relating to the key object by using peripheralframes sequentially adjacent to the main topic frame. Further, thecomposite image frame creator 120 creates a novel composite image frameby using the key object and the background object. Hereinafter, thecreation of the novel composite image frame will be described withreference to FIG. 2.

FIG. 2 is a view illustrating processes for creating the composite imageframe according to an exemplary embodiment of the present invention.Referring to FIG. 2, assuming that trees on an island in a sea isselected as the main topic, the key object and the background object arerespectively extracted by using the main topic frames 201 and 202, and aperipheral frame 203 sequentially adjacent to the two main topic frames201 and 202. Specifically, the key objects, which are included in aregion marked by squares 201 a and 202 a, are extracted from the maintopic frames 201 and 202, and then the background objects included inthe region marked by the squares 203 a and 203 b are estimated from theperipheral frame 203. Thereafter, the regions 201 a and 202 a, includingthe key objects, and the regions 203 a and 203 b, including thebackground objects, are integrated so as to create a novel compositeimage frames 204 and 205.

After the composite image frame is created by the above-mentionedmethod, a wide-angle image frame creator 130 creates an image framehaving common information to the composite image frame, and a wide-angleimage frame including the composite image frame, from the continuousimage frames. The creation of the wide-angle frame will be describedwith reference to FIG. 3.

FIG. 3 is a view illustrating processes for creating the wide-angleimage frame according to an exemplary embodiment of the presentinvention. Referring to FIG. 3, assuming that the appearance of vehiclesis selected as a main topic, the composite frames 301, 302 and 303respectively may become a composite image frame created by the method ofFIG. 2, or a general image frame including common informationsequentially and spatially adjacent to the composite image frame. Ifprocesses for creating the composite image frame are omitted and thewide-angle image frame is directly created, frames 301, 302, and 303 areregarded as the main topic frame, or the peripheral frames sequentiallyadjacent to the main topic frame.

In this exemplary embodiment, frame 301 is a composite image framecreated by the composite image frame creator 120, and the frames 302 and303 are an image frame including common information to the compositeimage frame 301, the three frames 301, 302 and 303 are interlinked toone another so as to create the wide-angle image frame 304.

In this case, in order to connect the frames 301, 302, and 303, theframes must have a relation among them. To this end, there is comparedthe relation between the object of the composite image frame and theobject of the image frame including common information to the compositeimage frame. In this exemplary embodiment of the present invention, inorder to determine this relation, the similarity of the area and thecolor of the objects are compared with one another. As a result ofcomparison, in the case where the similarity is greater than a desiredreference, it is determined that there is a relation between thecorresponding frames and, thus, it is possible to create the wide-angleimage frame 304. On the other hand, in the case where the similarity isless than the desired reference, as it is determined that there is norelation between the corresponding frames, the corresponding frame isexcluded from the wide-angle image frame. The reason for setting thedesired reference in order to compare the relation is to prevent thewide-angle frame from appearing as an unrealistic image.

After the wide-angle image frame 304 is created by the above-mentionedmethod, the unit 140 for setting an attractive region sets theattractive region, which a user intents to output, in the wide-angleimage frame 304. As shown in FIG. 3, the region 304 a marked by a dottedline is set as the attractive region. The attractive region 304 a is theportion which the user specially intends to output among the wide-angleimage frames expressing the main topic set by the user, and may includethe object mainly expressing the main topic. Meanwhile, the attractiveregion 304 a may be set while the window of adjustable size and positionmoves on the wide-angle image frame 304 in order to set the attractiveregion 304 a.

After the attractive region 304 a is set by the above-mentioned method,the attractive region image 305 is output by the output unit 150.

Further, image processing may be carried out in order to improve thequality of image, and the resolution of the image 305 may be adjusted tobe suitable for the size of the output image, before the attractiveregion image 305 is output. Specifically, the image processor 151processes the image suitable to be output with respect to the attractiveregion. For example, in the case of selecting the portrait as the maintopic, a smooth filtering may be applied to enhance the reappearance ofthe skin color. Further, a noise reduction algorithm may be applied toreduce noise, while an another algorithm may be applied to improvebrightness, saturation, and contrast.

Furthermore, a resolution controller 152 can control the resolution withrespect to the attractive region in which the image is processed. Forexample, when an image having a size of 4 inch×6 inch is about to beoutput as an image with the resolution of 600 dpi (dot per inch), theimage is converted into an image with the resolution of 2400×3600 pixelsize. An image interpolation or a super resolution can be used as amethod for increasing the resolution.

Hereinafter, the word “unit” used in the exemplary embodiments of thepresent invention as shown in FIG. 1 may include a hardware structuralelement such as software, Field Programmable Gate Arrays or ApplicationSpecific Integrated Circuits (ASIC), which carry out specifiedfunctions. However, the unit is not limited to those. The structuralelements may be stored in a storage medium to be reproduced on one ormore processors. Thus, as an example, the structural elements includestructural elements such as software structural elements,object-oriented software structural elements, class structural elements,and task structural elements, processes, functions, properties,procedures, sub-routines, segments of a program code, drivers, firmware,microcode, circuits, data, databases, data structures, tables, arrays,and parameters. Function provided by the structural elements may beintegrated by a small amount of structural elements, or separated intoadditional structural elements. Further, the structural elements arerealized so as to reproduce one or more CPUs in the device.

Hereinafter, the entire sequential processes carried out according to anexemplary embodiment of the present invention will be described withreference to FIG. 4. FIG. 4 is a flowchart illustrating processes forselectively outputting image frames.

First, the main topic frame extractor 110 extracts the main topic framein which the reflection ratio of the main topic selected by the user isgreater than the desired reference ratio from continuous image framesS110. Then, the composite image frame creator 120 extracts the keyobject expressing the main topic from the main topic frame, andestimates a background object with respect to the key object by usingperipheral frames sequentially adjacent to the main topic frame, therebycreating the composite image frame using the key object and thebackground object S120.

Next, the wide-angle image frame creator 130 creates the image framehaving common information to the composite image frame, and thewide-angle image frame including the composite image frame, from thecontinuous image frames S130. In this case, the area and color of theobject of the composite image frame are compared with those of theobject of the image frame including the common information to thecomposite image frame in order to determine if they have similarity. Asthe result of comparison, if the similarity is greater than the desiredreference, the wide-angle image frame can be created.

The attractive region setting unit 140 carries out setting of theattractive region with relation to the wide-angle image frame S140, andsets the attractive region on the wide-angle image frame by using awindow capable of adjusting a location and a size of the attractiveregion.

The output unit 150 performs outputting the attractive region S150. Atthis time, the image processor 151 carries out the image processingsuitable for outputting the attractive region, and the resolutioncontroller 152 can adjust the resolution with respect to the attractiveregion which is image-processed.

The exemplary embodiments of the present invention can be applied tohardcopy output apparatuses such as color laser printers, photoprinters, inkjet printers, and color multi-functional printers, softcopyoutput apparatuses such as plasma display panels (PDPs), liquid crystaldisplays (LCDs), and mobile display devices, and image storage andoutput apparatuses such image processing software, etc.

It would be obvious to persons killed in this art that the scope of thedevice for selectively outputting of the image frame according to thepresent invention pertains to computer-readable recording media havingprogram code recorded in order to carry out the above-mentioned methodin a computer.

According to the exemplary embodiment of the present invention, the maintopic frame, which reflects the best main topic set by the user amongthe image frames of the continuously photographed moving pictures, isdetected. Then, the images are integrated based on the detected maintopic frame so as to constitute the wide-angle image frame centeredaround the object, which reflects the main topic well, so that the userselects and outputs the attractive region, and thereby it is possible tooptimize a topic and region that a user is interested in, and to outputa high quality of images during the output of the moving pictures.

The effects of the present invention are not limited to theabove-mentioned effects, and other effects, which are not mentioned,will be apparent to a person skilled in the art.

Although exemplary embodiments of the present invention has beendescribed for illustrative purposes, those skilled in the art willappreciate that various modifications, additions and substitutions arepossible, without departing from the scope and spirit of the inventionas disclosed in the accompanying claims.

1. An apparatus for selectively outputting image frames, comprising: amain topic frame extractor which extracts a main topic frame, whichexpresses a main topic selected by a user, from continuous image frames;a composite image frame creator which extracts a key object to realizethe main topic from the main topic frames, and estimates a backgroundobject with respect to the key object to create a composite image frameusing the key object and the background object; a wide-angle image framecreator which creates a wide-angle image frame, which comprises thecomposite image frame and an image frame having common information withthe composite image frame, among the continuous image frames; anattractive region setting unit which sets an attractive region withrespect to the wide-angle image frame; and an output unit which outputsthe attractive region.
 2. The apparatus of claim 1, wherein thecomposite image frame creator estimates a background object with respectto a key object, by using a peripheral frame sequentially adjacent tothe main topic frame.
 3. The apparatus of claim 2, wherein the outputunit comprises an image processor which processes an image suitable foroutputting with respect to the attractive region, and a resolutioncontroller which adjusts a resolution of the image suitable foroutputting.
 4. The apparatus of claim 2, wherein the main topic isselected by the user from content expressed by the continuous imageframe, and comprises at least one of a portrait topic, a landscapetopic, an animal topic, and a structure topic.
 5. The apparatus of claim2, wherein the main topic frame extractor extracts the main topic framebased on a comparison of an area ratio or a color ratio of the keyobject to a reference ratio.
 6. The apparatus of claim 2, wherein thewide-angle image frame creator creates the wide-angled image frame basedon a comparison of an area and a color of an object of the compositeimage frame with those of an object of the image frame including commoninformation to the composite image frame to determine if there is asimilarity between the image frame and the composite image frame,wherein if the similarity is greater than a a set reference similarity,the composite image frame is interlinked to the image frame, so as tocreate the wide-angle image frame.
 7. The apparatus of claim 2, whereinthe attractive region setting unit sets the attractive region on thewide-angle image frame using a window configured to adjust a locationand a size of the attractive region.
 8. The apparatus of claim 3,wherein the image processor processes the images suitable for outputtingby filtering or carrying out a noise reduction algorithm on the imagesin order to improve the quality of the images depending oncharacteristics of the attractive region.
 9. A method of selectivelyoutputting image frames, comprising: (a) extracting a main topic frame,which expresses a main topic selected by a user, from continuous imageframes; (b) extracting a key object, which realizes the main topic, fromthe main topic frames, and estimating a background object with respectto the key object to create a composite image frame by using the keyobject and the background object; (c) creating a wide-angle image frameincluding the composite image frame and an image frame having commoninformation with the composite image frame, among the continuous imageframes; (d) setting an attractive region with respect to the wide-angleimage; and (e) outputting the attractive region.
 10. The method of claim9, wherein the background object is estimated by using a peripheralframe sequentially adjacent to the main topic frame.
 11. The method ofclaim 10, wherein (e) comprises (e1) processing an image suitable foroutputting with respect to the attractive region, and (e2) adjusting aresolution of the image suitable for outputting.
 12. The method of claim10, wherein the main topic is selected by the user from contentexpressed by the continuous image frames, and comprises at least one ofa portrait topic, a landscape topic, an animal topic, and a structuretopic.
 13. The method of claim 10, wherein (a) comprises extracting themain topic frame based on a comparison of an area ratio or a color ratioof the key object to a reference ratio.
 14. The method of claim 10,wherein in (c) the wide-angle frame is created by comparing an area anda color of an object of the composite image frame with those of anobject of the image frame including common information to the compositeimage frame to determine if there is a similarity between the imageframe and the composite image frame, wherein if the similarity isgreater than a set reference similarity, the composite image frame isinterlinked to the image frame so as to create the wide-angle imageframe.
 15. The method of claim 10, wherein (d) comprises setting theattractive region on the wide-angle image frame using a windowconfigured to adjust a location and a size of the attractive region. 16.The method of claim 11, wherein (e1) comprises filtering images in orderto improve a quality of the images depending on characteristics of theattractive region, or carrying out a noise reduction algorithm.
 17. Acomputer-readable recording medium having program code for carrying outthe method of claim 9 in a computer.